Papers with KV sharing
MLKV: Multi-Layer Key-Value Heads for Memory Efficient Transformer Decoding (2025.findings-naacl)
Copied to clipboard
| Challenge: | Multi-Layer Key-Value (MLKV) sharing reduces memory usage by 6x compared to Multi-Query Attention and Grouped-Query Attributes. |
| Approach: | They propose a novel approach that extends KV sharing across transformer layers to reduce memory usage beyond what was possible with Multi-Query Attention and Grouped-Query Attributes. |
| Outcome: | The proposed approach reduces KV cache size by 6x with minimal performance loss and scales linearly with model size, batch size, and sequence length. |